Some robots and/or robot control systems utilize one or more techniques to detect and/or localize various objects in the robot's environment. For example, techniques have been proposed for utilizing vision data (e.g., point cloud data) captured by a vision component (e.g., camera, laser scanner) of a robot to classify objects in an environment of the robot and to estimate a pose (position and orientation) of the object in the environment. The classification of an object according to such techniques classifies the entire object into a particular category selected from a taxonomy of categories.